Malaria Detection

Problem Definition

The context:

Why is this problem important to solve?

The objectives:

What is the intended goal?

The key questions:

What are the key questions that need to be answered?

The problem formulation:

What is it that we are trying to solve using data science?

Mounting the Drive

Loading libraries

Let us load the data

Data Description

There are a total of 24,958 train and 2,600 test images (colored) that we have taken from microscopic images. These images are of the following categories:

Parasitized: The parasitized cells contain the Plasmodium parasite which causes malaria
Uninfected: The uninfected cells are free of the Plasmodium parasites

The files have been extracted to the local session of Google Colab. The extracted folder would have the following structure:

Folder_Structure.PNG

The extracted folder has different folders for train and test data which further contains the different sizes of images for parasitized and uninfected cells within the respective folder name.

The size of all images is scaled to 64x64 and are converted to 4D arrays so that they can be used as an input for the convolutional neural network. Also, we have created the labels for both types of images to be able to train and test the model.

Important thing to notice - we have read all our data in the variables of this Data class:

dataclass

Visualizing training data

Observations from training data

Visulaizing test data

Observations from test data

Overall observations looking at the images from train and test data

Data balance for training data

Observations about data balance for training data

Data balance for test data

Observations about data balance for test data

Mean Image for training and test data

Lets have a look at the mean images if both training and test data

Observations from mean Image for both training and test data

Data Normalization:

Observations for data normalization:

RGB to HSV

Observations from HSV data

Different types of data that we will be using to train our models

Classes to organize data

Different types of models that we will have

Classes to organize models definitions

Models definition and model data permutations for form final models

To train final models we will be training each of the 5 model definitions with 4 different types of data separately. That will give us 4x5 = 20 different types of models.

AbstractModel class for performing common functions

Base Model

Base model | RGB data

Observations for Base model | RGB data:

Base model | RGB data | Data augmentation

Observations for Base model | RGB data | Data augmentation:

Base model | HSV data

Observations for Base model | HSV data

Base model | HSV data | Data augmentation

Observations for Base model | HSV data | Data augmentation

Model With Leaky Relu

Model With Leaky Relu | RGB data

Observations for Model With Leaky Relu | RGB data

Model With Leaky Relu | RGB data | Data augmentation

Observations for Model With Leaky Relu | RGB data | Data augmentation

Model With Leaky Relu | HSV data

Observations for Model With Leaky Relu | HSV data

Model With Leaky Relu | HSV data | Data augmentation

Observations for Model With Leaky Relu | HSV data | Data augmentation

Model With Leaky Relu and Batch Normalization

Model With Leaky Relu & Batch Normalization | RGB data

Observations for Model With Leaky Relu & Batch Normalization | RGB data

Model With Leaky Relu & Batch Normalization | RGB data | Data augmentation

Observations for Model With Leaky Relu & Batch Normalization | RGB data | Data augmentation

Model With Leaky Relu & Batch Normalization | HSV data

Observations for Model With Leaky Relu & Batch Normalization | HSV data

Model With Leaky Relu & Batch Normalization | HSV data | Data augmentation

Observations for Model With Leaky Relu & Batch Normalization | HSV data | Data augmentation

Model With Transfer Learning using VGG16

Model With Transfer Learning VGG16 | RGB data

Observations for Model With Transfer Learning VGG16 | RGB data

Model With Transfer Learning VGG16 | RGB data | Data augmentation

Observations for Model With Transfer Learning VGG16 | RGB data | Data augmentation

Model With Transfer Learning VGG16 | HSV data

Observations for Model With Transfer Learning VGG16 | HSV data

Model With Transfer Learning VGG16 | HSV data | Data augmentation

Observations for Model With Transfer Learning VGG16 | HSV data | Data augmentation

Creating model data objects referening data with images having bigger size of size 75x75

Model With Transfer Learning using Inception V3

Model With Transfer Learning Inception V3 | RGB data

Observations for Model With Transfer Learning Inception V3 | RGB data

Model With Transfer Learning Inception V3 | RGB data | Data augmentation

Observations for Model With Transfer Learning Inception V3 | RGB data | Data augmentation

Model With Transfer Learning Inception V3 | HSV data

Observations for Model With Transfer Learning Inception V3 | HSV data

Model With Transfer Learning Inception V3 | HSV data | Data augmentation

Observations for Model With Transfer Learning Inception V3 | HSV data | Data augmentation

Measures of success for the models

Comparison of different models

model-comparison

Some observations:

Best model amonst all the above listed models

twobest

Difference between the models:

Appendix

Complete Class diagram for the python solution

classes